Of : Treelets — an Adaptive Multi - Scale Basis for Sparse Unordered Data

We congratulate Lee, Nadler and Wasserman (henceforth LNW) on a very interesting paper on new methodology and supporting theory. Treelets seem to tackle two important problems of modern data analysis at once. For datasets with many variables, treelets give powerful predictions even if variables are highly correlated and redundant. Maybe more importantly, interpretation of the results is intuitive. Useful insights about relevant groups of variables can be gained. Our comments and questions include: (i) Could the success of treelets be replicated by a combination of hierarchical clustering and PCA? (ii) When choosing a suitable basis, treelets seem to be largely an unsupervised method. Could the results be even more interpretable and powerful if treelets would take into account some supervised response variable? (iii) Interpretability of the result hinges on the sparsity of the final basis. Do we expect that the selected groups of variables will always be sufficiently small to be amenable for inter-pretation? 1. Treelets or hierarchical clustering combined with PCA. A main part of the treelet algorithm achieves two main objectives: (1) Variables are ordered in a hierarchical scheme. Highly correlated variables are typically " close " in the hierarchy. (2) A basis on the tree is chosen. Each node of the tree is associated with a " sum " (and also a " difference " variable). Clearly, treelets are more elegant than any method trying to achieve these two goals separately. As LNW write in Section 1: " The novelty and contribution of our approach is the simultaneous construction of a data-driven multi-scale orthogonal basis and a hierarchical cluster tree. " We are left wondering, though, how different treelets are to the following scheme. First, variables are ordered in a hierarchical clustering scheme—for concreteness,